Firstly, we test the memorization ability and its influence of our HGMM on several small size datasets. The memory influence functions (i.e., formations (12), (13), (14) and (15)) were preloaded in our HGMM and evaluated by the m-fold cross validation (i.e., level-one-out validation, LOO for short). We set the baseline by setting the memory influence function be an identity matrix which is actually L2 loss SVM with decision (7) according to Theorem 4.3 (ii). Table II reports their highest LOO training and testing accuracies. From Table II, it is observed that our HGMM with either memory influence function has 100% training accuracies on all of these datasets.

Classifying the training data correctly without over-fitting is one of the goals in machine learning. In this paper, we propose a generalization-memorization mechanism, including a generalization-memorization decision and a memory modeling principle. Under this mechanism, error-based learning machines improve their memorization abilities of training data without over-fitting. Specifically, the generalization-memorization machines (GMM) are proposed by applying this mechanism. The optimization problems in GMM are quadratic programming problems and could be solved efficiently. It should be noted that the recently proposed generalization-memorization kernel and the corresponding support vector machines are the special cases of our GMM. Experimental results show the effectiveness of the proposed GMM both on memorization and generalization.